Dictionary Organization in Linguistic Automaton for Oriental Languages

The central problem for natural language processing (NLP) systems dealing with non-Indo-European (“Oriental”) languages is how to develop automatic dictionaries (AD) and dictionary entry (DE) schemes. The point is that the need of Oriental language industrial NLP has been felt for some time. It has acquired additional urgency with the rapid growth of business contacts between Russia and the nations of the Middle East and the Pacific Rim. The very notions of such language items as root, stem, word form (w/f) and text word (t/w), which are so essential in designing an AD, are quite distinct in each of the Oriental languages and fundamentally different from what we are used to treat as a root, a t/w etc. in the Indo-European languages. If an Oriental language AD is to be integrated into a multimodular linguistic automaton and the system has to retain its basic structure, this project requires development of various forms of sub-lexicon databases. The structure of Arabic and Hebrew t/w requires elaboration of four versions of DE while the differentiation of full and structural words in Chinese provides two versions. An agglutinative word structure model, such as Turkic and Finno-Ugric, requires a tree-structured database and special procedures of access.